Move DagBag to airflow/dag_processing by ephraimbuddy · Pull Request #55139 · apache/airflow

ephraimbuddy · 2025-09-01T12:20:55Z

DagBag is not a DB model, it’s the parser/collector used by the dag processor. This change moves it to a more natural home under airflow/dag_processing, reduces confusion between “DBDagBag” and "DagBag",
and helps avoid import tangles.

What changed:

New module: airflow.dag_processing.dagbag now hosts: DagBag, _capture_with_reraise, FileLoadStat, timeout, sync_bag_to_db

Old location airflow.models.dagbag:
Trimmed to DB-facing helpers (DBDagBag, etc.)

Adds getattr shim for backward compatibility.

Updated imports across the codebase

Pre-commit path allowlist updated to include the new file and remove the old path.

Deprecation & migration

Deprecated: from airflow.models.dagbag import DagBag

Use instead: from airflow.dag_processing.dagbag import DagBag

A deprecation warning is emitted via the shim; no functional behavior change intended.

Notes

No runtime logic changes to parsing or DAG discovery.

Tests and CLI code updated to the new import path.

sync_bag_to_db moved alongside DagBag to keep parsing + persistence in one place.

jason810496

This makes sense to me.

airflow-core/src/airflow/dag_processing/bundles/manager.py

amoghrajesh

The change makes sense to me

airflow-core/src/airflow/cli/commands/dag_command.py

airflow-core/src/airflow/dag_processing/bundles/manager.py

airflow-core/src/airflow/utils/cli.py

o-nikolas · 2025-09-24T18:34:58Z

I'm just running into this PR now since it causes merge conflicts with one of mine. Should we not move the test module airflow-core/tests/unit/models/test_dagbag.py to airflow-core/tests/unit/dag_processing/test_dagbag.py? @ephraimbuddy @jedcunningham ?

jedcunningham · 2025-09-24T18:48:44Z

Yep, it should, good call.

* Move DagBag to airflow/dag_processing DagBag is not a DB model, it’s the parser/collector used by the dag processor. This change moves it to a more natural home under airflow/dag_processing, reduces confusion between “DBDagBag” and "DagBag", and helps avoid import tangles. What changed: New module: airflow.dag_processing.dagbag now hosts: DagBag, _capture_with_reraise, FileLoadStat, timeout, sync_bag_to_db Old location airflow.models.dagbag: Trimmed to DB-facing helpers (DBDagBag, etc.) Adds __getattr__ shim for backward compatibility. Updated imports across the codebase Pre-commit path allowlist updated to include the new file and remove the old path. Deprecation & migration Deprecated: from airflow.models.dagbag import DagBag Use instead: from airflow.dag_processing.dagbag import DagBag A deprecation warning is emitted via the shim; no functional behavior change intended. Notes No runtime logic changes to parsing or DAG discovery. Tests and CLI code updated to the new import path. sync_bag_to_db moved alongside DagBag to keep parsing + persistence in one place. * fixup! Move DagBag to airflow/dag_processing * fixup! fixup! Move DagBag to airflow/dag_processing * fixup! fixup! fixup! Move DagBag to airflow/dag_processing * fixup! fixup! fixup! fixup! Move DagBag to airflow/dag_processing * fixup! fixup! fixup! fixup! fixup! Move DagBag to airflow/dag_processing * fixup! fixup! fixup! fixup! fixup! fixup! Move DagBag to airflow/dag_processing * fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! Move DagBag to airflow/dag_processing * Remove get_dagbag method from DagBundlesManager * Move DagBag to airflow/dag_processing DagBag is not a DB model, it’s the parser/collector used by the dag processor. This change moves it to a more natural home under airflow/dag_processing, reduces confusion between “DBDagBag” and "DagBag", and helps avoid import tangles. What changed: New module: airflow.dag_processing.dagbag now hosts: DagBag, _capture_with_reraise, FileLoadStat, timeout, sync_bag_to_db Old location airflow.models.dagbag: Trimmed to DB-facing helpers (DBDagBag, etc.) Adds __getattr__ shim for backward compatibility. Updated imports across the codebase Pre-commit path allowlist updated to include the new file and remove the old path. Deprecation & migration Deprecated: from airflow.models.dagbag import DagBag Use instead: from airflow.dag_processing.dagbag import DagBag A deprecation warning is emitted via the shim; no functional behavior change intended. Notes No runtime logic changes to parsing or DAG discovery. Tests and CLI code updated to the new import path. sync_bag_to_db moved alongside DagBag to keep parsing + persistence in one place. * fixup! fixup! fixup! fixup! fixup! fixup! fixup! Move DagBag to airflow/dag_processing * Remove get_dagbag method from DagBundlesManager * Fix dagbag mock * import DagBag from models instead of models.dagbag in providers * fixup! import DagBag from models instead of models.dagbag in providers * fix fab www-hash * fixup! fix fab www-hash * leave DagBag import to be from airflow.models.dagbag in init so it issues deprecation warning * Use DBDagBag in fab type checking if in AF3.1+ * fix static checks * Update fab hash * fixup! Update fab hash * fixup! fixup! Update fab hash * fixup! fixup! fixup! Update fab hash

amoghrajesh · 2026-02-11T08:01:25Z

cc: @atul-astronomer flagging for test plan. No functional change here but deprecation can be tested

ephraimbuddy requested review from XD-DENG, amoghrajesh, ashb, bugraoz93, jason810496, jedcunningham, kaxil, pierrejeambrun, rawwar and shubhamraj-git as code owners September 1, 2025 12:20

boring-cyborg bot added area:API Airflow's REST/HTTP API area:CLI area:DAG-processing area:dev-tools area:Scheduler including HA (high availability) scheduler area:serialization area:task-sdk backport-to-v3-1-test Mark PR with this label to backport to v3-1-test branch labels Sep 1, 2025

ephraimbuddy force-pushed the move-dagbag-to-dag-processing branch from a690dbb to 4689218 Compare September 1, 2025 12:21

ephraimbuddy removed the backport-to-v3-1-test Mark PR with this label to backport to v3-1-test branch label Sep 1, 2025

ephraimbuddy added this to the Airflow 3.1.0 milestone Sep 1, 2025

jason810496 reviewed Sep 1, 2025

View reviewed changes

airflow-core/src/airflow/dag_processing/bundles/manager.py Outdated Show resolved Hide resolved

ephraimbuddy force-pushed the move-dagbag-to-dag-processing branch 2 times, most recently from fcb54e6 to 7e723c1 Compare September 2, 2025 07:25

amoghrajesh reviewed Sep 2, 2025

View reviewed changes

ephraimbuddy force-pushed the move-dagbag-to-dag-processing branch 2 times, most recently from 99391eb to 8486677 Compare September 2, 2025 12:44

jedcunningham reviewed Sep 2, 2025

View reviewed changes

airflow-core/src/airflow/cli/commands/dag_command.py Outdated Show resolved Hide resolved

airflow-core/src/airflow/dag_processing/bundles/manager.py Outdated Show resolved Hide resolved

ephraimbuddy force-pushed the move-dagbag-to-dag-processing branch from 0fab14c to 4ffaa8d Compare September 3, 2025 08:24

ephraimbuddy commented Sep 3, 2025

View reviewed changes

airflow-core/src/airflow/utils/cli.py Show resolved Hide resolved

ephraimbuddy added 2 commits September 24, 2025 09:55

fixup! Update fab hash

7fc31de

fixup! fixup! Update fab hash

9cba1e9

ephraimbuddy force-pushed the move-dagbag-to-dag-processing branch from 16f35bf to 9cba1e9 Compare September 24, 2025 08:55

fixup! fixup! fixup! Update fab hash

883a0d4

ephraimbuddy merged commit 12b8e4c into apache:main Sep 24, 2025
109 checks passed

ephraimbuddy deleted the move-dagbag-to-dag-processing branch September 24, 2025 12:09

simi mentioned this pull request Sep 24, 2025

Add bundle_path temporarily to sys.path during DagBag parsing. #55894

Merged

eladkal mentioned this pull request Sep 25, 2025

Status of testing Providers that were prepared on September 25, 2025 #56091

Closed

19 tasks

kaxil mentioned this pull request Sep 25, 2025

Run providers compat test with Airflow 3.1 #56118

Merged

eladkal mentioned this pull request Sep 28, 2025

Status of testing Providers that were prepared on September 28, 2025 #56183

Closed

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Comments

Move DagBag to airflow/dag_processing#55139

Move DagBag to airflow/dag_processing#55139
ephraimbuddy merged 24 commits intoapache:mainfrom
astronomer:move-dagbag-to-dag-processing

ephraimbuddy commented Sep 1, 2025 •

edited

Loading

Uh oh!

jason810496 left a comment

Uh oh!

Uh oh!

amoghrajesh left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

o-nikolas commented Sep 24, 2025

Uh oh!

jedcunningham commented Sep 24, 2025

Uh oh!

amoghrajesh commented Feb 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Comments

Conversation

ephraimbuddy commented Sep 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changed:

Deprecation & migration

Notes

Uh oh!

jason810496 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

amoghrajesh left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

o-nikolas commented Sep 24, 2025

Uh oh!

jedcunningham commented Sep 24, 2025

Uh oh!

amoghrajesh commented Feb 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

ephraimbuddy commented Sep 1, 2025 •

edited

Loading